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CRITERION-REFERENCED  SYSTEM  APPROACH  TO  EVALUATION 
OF  COMBAT  UNITS 

/ 

Systems  engineering  of  training  and  its  subsidiary  criterion- 
referenced  measurement  have  been  invaluable  tools  for  increasing  the 
job-relevance  of  military  training  and  evaluation.  These  tools  have 
provided  an  indispenscible  point  of  departure  and  a  framework  for  en¬ 
suring  accountability.  However,  they  have  been  developed  within  the 
context  of  relatively  simple,  procedural  tasks  that  are  necessary  but 
not  always  sufficient  for  describing  jobs  as  performed  in  working  en¬ 
vironments.  The  tools  work  comfortaibly  for  hard,  individual  skills. 

But  the  o^ection  often  heard  is  that  the  soft  skills  have  yet  to  be 
attacked  successfully  with  those  tools.  Those  of  us  who  have  moved  the 
focus  of  our  evaluation  research  from  the  school  setting  to  the  combat 
unit  are  especially  sensitive  to  this  objection  because  we  face  the 
added  complication  of  two-sided,  tactical,  collective  behavior. 

To  improve  the  training  and  evaluation  of  such  behavior,  the  Army 
Research  Institute  for  the  Behavioral  Sciences  (ARI)  has  been  pursuing 
research  on  tactical  engagement  simulation.  In  addition,  we  have  been 
developing  a  supporting  system  of  evaluation.  This  paper  deals  with  re¬ 
search  on  the  evaluation  system;  first,  let  us  briefly  describe  the 
engagement  simulation  test  bed.^ 

Engagement  simulation  (ES)  currently  is  a  set  of  techniques  for  con¬ 
ducting  real-time,  two-sided  free  play,  tactical  exercises  at  the  com¬ 
bined  arms  reinforced  platoon  level.  One  of  its  key  features  is  a  set 
of  objective,  casualty  assessment  methods  that  allow  almost  real-time 
feedback  to  participants.  For  example,  a  rifleman  can  fire  at  a  target 
and  register  a  hit  by  calling  out  a  number  on  the  helmet  of  the  opposing 
infantryman.  A  tank  gunner  can  similarly  register  a  hit  against  another 
tank.  Kills  are  relayed  via  radio  by  a  controller  to  a  net  control  sta¬ 
tion,  which  in  turn  radios  the  target  that  it  is  out  of  action.  Buyable 
pyrotechnics  add  visual  cues  and,  therefore,  realism  to  the  battle.®  With 
these  and  other  techniques  for  artillery  and  antitank  weapons,  it  is  pos¬ 
sible  to  measure  casualties  over  time  and  thereby  provide  for  objective 
assessment  of  the  outcomes  of  tactical  performance. 


^Paper  originally  presented  at  the  19th  Military  Testing 
Meeting,  San  Antonio,  Tex.,  19  October  1977. 
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Training  Circular  71-5.  REALTRAIN:  Tactical  Training 
Arms  Elements.  U.S.  Army  Armor  School/U.S.  Army  Research  Institute, 
January  1975. 


Several  years  ago,  when  engagement  simulation  was  developed  as  a 
training  methodology,  its  developers  felt  that  the  evaluation  problem 
for  unit  training  had  been  solved,  because  objective  measures  of  casual¬ 
ties  were  now  available.  What  else  was  needed?  However,  an  alternative 
view  held  that  a  great  deal  else  was  needed:  that  the  engagement  simula¬ 
tion  test  bed  had  opened  up  a  Pandora's  box  with  respect  to  the  measure¬ 
ment  and  interpretation  of  unit  combat  performance.  This  alternative 
view  argued  for  a  system  of  evaluation  which  included,  at  least,  concern 
for  process  measurements  and  a  scheme  for  uncovering  the  patterns  and 
relationship  among  these  two  sets  of  measures ,  taken  continually  through 
a  training  exercise,  plus  other  major  features. 

This  discussion  outlines  what  an  adequate  system  of  evaluation  might 
look  like  for  the  engagement  simulation  test  bed  and  mentions  some  of  our 
research  experiences  with  various  pcirts  of  the  system  (see  Figure  1) . 

To  proceed  logically  and  efficiently  through  the  development  of  an  en¬ 
gagement  simulation  relevant  evaluation  system,  we  begin  with  the  develop¬ 
ment  of  a  model  or  model (s)  to  define 

•  Measurement  concepts , 

•  Data  processing  concepts,  and 

•  Data  interpretation  concepts . 

These  concepts  are  consistent  with  the  purposes  for  measuring  and  assess¬ 
ing  performance  in  the  first  place.  It  is  at  this  point  that  evaluation 
aims  and  philosophical  biases  can  be  put  on  the  line.  If  the  major  pur¬ 
pose  of  evaluation  is  diagnostic  feedback  in  support  of  a  training  sys¬ 
tem,  that  purpose  can  be  made  explicit  and  the  rest  of  the  system 
designed  accordingly.  This  last  statement  may  seem  obvious  and  self- 
evident,  but  in  practice  it  may  not  be  so  obvious.  One  of  the  philo-  ^ 
sophical  problems  with  the  Army  Training  and  Evaluation  Program  (ARTEP) 
is  that  it  does  not  adequately  distinguish  between  evaluation  for  train¬ 
ing  diagnosis  and  evaluation  for  accountability.  A  result  has  been  that 
many  commanders  regard  ARTEP  as  a  report  card  in  spite  of  guidance  to 
the  contrary  from  the  Army  Training  and  Doctrine  Command  (TRADOC) .  This 
observation,  which  came  out  of  a  current  ARI  study,  suggests  at  least 
one  fundamental  problem  with  ARTEP  as  a  training  model. ^ 


^ARTEP  71-2.  Army  Training  and  Evaluation  Program  for  Mechanized  In¬ 
fantry/Tank  Task  Force,  June  1977. 
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Human  Sciences  Research,  Inc.  Interim  Report  (Revised) ,  Improved 
Army  Training  and  Evaluation  Program  (ARTEP)  Methods  for  Unit  Evalua¬ 
tion,  21  October  1977. 
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Steps  in  development  of  system: 

1.  Develop  a  model  or  model (s)  to  define: 

•  Measurement  concepts 

•  Data  processing  concepts 

•  Data  interpretation  concepts 

2.  Define  data  requirements  and  develop  processing 
methods 

3.  Define  and  develop  data  collection  methods 

4.  Define,  develop,  performance  benchmarking 
techniques ,  i .  e . ,  standards . 


Figure  1.  Engagement  simulation  evaluation  system. 


The  next  step  in  system  development  would  be  to  define  the  data 
requirements  and  data  processing  methods  that  are  needed  to  fit  the 
model  or  model(s)  constructed  in  step  1.  If,  for  example,  in  step  1 
you  decide  that  information  about  patterns  of  tactical  movement  is  use¬ 
ful  for  diagnostic  purposes,  that  would  suggest  a  need  to  know  what 
fire  elements  are  where  and  when.  You  would  need  to  go  further  and 
decide  how  much  information  on  position  location  is  needed  and  how 
accurate  it  needs  to  be. 

Now  you  face  step  3,  which  requires  that  you  define  the  methods 
for  collecting  the  data  identified  in  step  2.  If  you  are  not  yet  famil¬ 
iar  with  the  realities  of  collecting  objective  performance  data  under 
field  operational  conditions,  you  would  soon  learn  about  them  at  this 
stage  of  system  development. 

Finally,  step  4  defines  the  performance  benchmarks  or  standards 
that  make  your  system  criterion-referenced.  This  is  probably  the  most 
difficult  step  of  all.  It  has  been  sidestepped  to  a  large  extent  by 
ARTEP  through  the  use  of  expressions  such  as  "Casualties  shall  not  be 
excessive,"  with  the  definition  of  the  benchmark  being  left  to  the 
evaluation  team.  ARTEP  has  also  sidestepped  the  criterion  issue  by 
using  mostly  procedural  standards,  which  are  at  a  more  global  level 
than  those  in  the  old  Army  Training  Tests  but  which  are  still 
procedures -based . 

We  can  now  review  some  areas  of  progress  that  ARI  has  made  in  con¬ 
tributing  to  such  a  criterion-referenced  system. 

1.  Modeling.  As  part  of  a  long-term  effort  to  validate  engagement 
simulation  training,  new  experimental  versions  of  ARTEP  are  being  pro¬ 
duced.  These  were  produced  specifically  for  some  developmental  tests 
at  Fort  Carson,  Colo.,  in  January  1978.  Accordingly,  they  were  de¬ 
signed  for  reinforced  platoon  missions,  i.e.,  for  tank  platoons  with 
supporting  infantry  squads  and  tube-launched,  optically  tracked,  wire- 
guided  (TOW)  missiles. 

For  those  not  familiar  with  ARTEPs,  Figure  2  shows  a  typical  page, 
from  ARTEP  7-45  FOR  MECHANIZED  INFANTRY  AND  COMBINED  ARMS  TASK  FORCE. 
Look  particularly  at  the  column  labeled  Training/Evaluation  Standards . 
Phrases  like  "Coordination. . .will  support...,"  "must  be  responsive," 
"without  sustaining  excessive  casualties,"  place  a  substantial  respon¬ 
sibility  on  the  evaluator.  In  Figure  3,  a  roughly  comparable  version 
of  an  engagement  simulation  ARTEP,  you  can  see  at  least  two  major 
revisions  in  ARTEP  concept. 

•  The  standards  and  the  rating  columns  have  been  eliminated.  In 
their  places  are  a  performance  data  and  results  section.  The 
measures  are  quantitative:  time,  range,  casualties. 
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UNIT:  Company /Teams 


Deliberate 

MISSION:  Daylight  Attack 


TASK 


CONDITIONS  TRAINING/EVALUATION  STANDARDS 


Eliminate  enemy 
resistance 


Enemy  engages  lead  ele¬ 
ments  of  the  TF  when 
they  are  within  anti¬ 
tank  range  of  his  posi¬ 
tion  (e.g.,  1,000  - 
2,000  meters) . 


A.  Coordination  of  mortar, 
machinegun,  antitank,  and 
nonorganic  fire  support 
will  support  the  scheme 

of  maneuver  (i.e.,  fire 
must  begin,  be  shifted, 
and  be  stopped  as  specified 
in  the  order  and  must  be 
responsive  to  requests 
from  maneuver  platoons) . 

B .  Company/teeim  elements 
use  proper  fire  and  maneu¬ 
ver  techniques  to  elimi¬ 
nate  enemy  resistance. 
Objective  is  secured  with¬ 
out  sustaining  excessive 
casualties  and  vehicle 
losses.  (Evaluator 
judgment . ) 


Figure  2.  Training  and  evaluation  outline. 


UNIT:  Company /Team 


MISSION:  Attack 


TASK/SUBTASK 


CONDITIONS  PERFORMANCE  DATA  RESULTS 


Execute  the  attack  (tanks)  Enemy  force 

defends 


Cross  the  LD 


Time  tanks  cross  LD 


Tanks  move  by  covered  and 
concealed  route  to  attack 


Tank  casualties  during 
movement 


Tanks  detect  the  enemy  at 
maximum  range 


Time  of  detection 
Detection  range 


Tanks  engage  the  enemy  at 
maximum  range 


Engagement  range 


Tanks  engage  the  enemy 


Casualties :  Opposition  Ovm 


Execute  the  attack  (INFANTRY) 


Key  personnel 
Personnel 
Weapons  disabled 
Vehicles  immobilized 


Figure  3.  Training  and  evaluation  outline. 
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•  Major  objectives  have  been  further  analyzed  into  intermediate 
objectives.  For  example,  the  task  of  eliminating  enemy  resis¬ 
tance  has  been  analyzed  into  the  weapons  systems  involved  and 
then  further  broken  down  into  weapon  systems  subtasks. 

What  has  happened  to  the  standards?  This  particular  model  of  ARTEP 
candidly  admits  that  we  don't  know  how  to  handle  the  standards  problem 
yet  and  moves  the  problem  to  one  side,  until  scientific  progress  in  this 
area  provides  some  useful  methodology.  Emphasis  shifts  here  away  from 
GO/NO-GO  type  of  evaluation.  Emphasis  is  placed  instead  on  obtaining  a 
rich,  detailed  description  of  the  behaviors  involved  in  two-sided  combat. 

That  emphasis  leads  to  two  essential  questions.  The  first  is  "What 
patterns  of  behavior  can  we  extract  from  the  various  performance  measures 
which  will  have  diagnostic  value?"  We  have,  for  excimple,  a  particular 
interest  in  showing  the  connection  or  correlations  among  tactical  move¬ 
ments,  processes  such  as  first  enemy  detections  and  outcome  measures 
such  as  casualties  inflicted  or  sustained. 

The  second  question  is,  "What  performance  tradeoffs  can  we  identify 
and  measure?"  A  commander  may  deliberately  sacrifice  cover  and  conceal¬ 
ment  in  order  to  fight  more  aggressively  or  move  more  quickly  toward  some 
tactical  objective.  The  significant  and  diagnostically  useful  measure¬ 
ment  concept  would  be  risk-taking  behavior,  instead  of  just  cover  and 
concealment . 

Again  we  ask  "What  has  happened  to  standards?"  They  have  not  been 
forgotten.  Until  the  standards  problem  is  solved,  an  evaluation  system 
is  not  criterion-referenced.  But  we  have  concluded  that  some  imaginative 
and  fresh  thinking  is  required,  along  with  supporting  research.  The  con¬ 
cept  which  we  are  currently  working  on  can  be  described  as  situation- 
specific  forecasting  of  the  dynamics  of  an  engagement  simulation  exer¬ 
cise,  along  with  various  tactical  processes  and  outcomes  such  as 
casualties . 

2 .  Data  Requirements .  The  second  ^tep  in  developing  an  evaluation 
system  is  to  define  performance  data  requirements  and  data  processing 
techniques.  The  modeling  (step  1)  can  provide  the  general  guidance  for 
the  second  step.  But  more  specifically,  our  approach  has  been  to  iden¬ 
tify  essential  elements  of  analysis  (EEA)  and  then  to  produce  measures 
of  effectiveness  (MOE)  by  phase  of  combat.  This  is  consistent  with  the 
ES  ARTEP  model.  Under  contract  to  ARI ,  Human  Systems,  Inc.  (HSI) ,  has 
generated  a  computer  listing  of  EEA's  along  with  methods  for  coding, 
processing,  and  displaying  the  results  of  a  computer  analysis  of  a 
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tactical  map.  Figure  4  shows  the  initial  list  that  HSI  generated. 

What  the  data  file  does,  in  effect,  is  to  describe  and  display  the  tac¬ 
tical  movements  of  two  opposing  combat  teams ,  Alpha  and  Bravo ,  involved 
in  an  ES  exercise.  The  list  indicates  which  fire  elements  are  in  what 
locations,  at  what  time,  what  the  terrain  is  like,  whether  or  not  there 
are  targets  of  opportunity,  and  what  casualties  result  from  direct  and 
indirect  fire. 

These  EEA's  were  put  together  in  various  ways  to  provide  MOE's 
for  each  of  the  phases  of  reinforced  platoon  attack  mission.  This 
mission  was  the  basis  for  a  developmental  test  at  Fort  Carson  in  Janu¬ 
ary  1978.  The  phases  were 

1.  Preparation  (i.e..  Planning), 

2.  Pre-Engagement  (i.e..  Movement  to  Contact), 

3.  Engagement  (Hostilities) ,  and 

4.  Post-Engagement  (Post-Attack  Security) . 

Some  examples  of  measures  of  effectiveness  are  shown  in  Figures  5 
and  6.  Figure  5  shows  an  MOE  for  Tactical  Formation  in  the  Pre- 
Engagement  Phase.  Figure  6  shows  an  example  from  the  Engagement  Phase. 
In  this  case,  an  MOE  for  enemy  detection  is  illustrated. 

The  methods  for  processing  data  in  the  HSI/ARI  data  file  are  some¬ 
what  constrained  by  their  small  quantity.  Tactical  exercises  (unlike 
most  individual  tasks)  require  several  hours  to  whole  days  to  complete 
and  are  very  costly  in  manpower  and  supplies.  Consequently,  data  are 
relatively  scarce  and  do  not  readily  lend  themselves  to  sophisticated 
multivariate  analysis.  Therefore,  until  a  large  data  base  is  built  up, 
we  will  probably  not  be  able  to  do  much  beyond  cross-tabulations  of  fre¬ 
quency  counts.  This  is  the  tack  we  have  been  taking  so  far.  Such  a 
tack  is  consistent  with  our  near-term  goal,  which  is  to  build  up  expe¬ 
rience  with  objective  measurement  of  tactical  exercises  and  to  learn 
which  measures  are  most  useful  for  diagnosis  of  training  deficiency. 


^Hansen,  D.  N.  &  Drewfs.  Small  Unit  Data  Input  Structure  and  Graphic 
Support  System.  Interim  Report.  Human  Systems,  Inc.,  28  June  1977. 


1 .  Team  membersh  tp 

2.  Trajectory  force  mix 

3.  Trajectory  identification  number 

4.  Trajectory  path  hex  number 

5.  Elapsed  exercise  time 

6.  Hex  terrain  type 

7.  Trajectory  mode 

8 .  Communications 

9.  Trajectory  plan  versus  actual  hex  position 

10.  Trajectory  overwatch  placements 

11.  Trajectory  potential  fire  fan  and  coverage  fire  fan 

12.  Trajectory  movement  rates 

13.  Target  of  opportunity  index 

14.  Attack  effectiveness  (direct  fire  effects) 

15.  Incoming  indirect  fire  effects 


Figure  4.  EEA's  being  addressed  by  engagement  simulation 
data  file. 
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Exercise  Phase: 


Pre -Engagement 


Measurement 


MOE: 


( 


EEA's: 


Concept:  Tactical  Formation 

What  number  of  sub-deployments  (partitions)  are  ordered 
and/or  initiated  in  term  movement  to  contract? 

What  specific  sub-deployments  of  what  specific  force 
mixtures  were  made  where,  when,  and  maintained  for  how 
long?  Under  what  conditions? 

Group  identifiers,  trajectory-tributary  ntomber,  force 
mixture,  team  identifier,  hex  number,  and  time. 


Figure  5.  Examples  of  measures  of  effectiveness. 
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Exercise  Phase: 


Engagement 


Measurement 


MOE: 


EEA's: 


Concept;  Detection  of  Enemy  Forces 

What  targets  were  detected  by  whom,  when,  in  what  sub-unit 
having  what  force  mix,  in  what  sub-deployed  partition, 
headed  where? 


Target  type. 

Target  number. 
Detector  type. 
Detector  number. 

Time, 

Locations  of  target. 
Location  of  detector. 


Figure  6.  Exeunple  of  measure  of  effectiveness 


3.  Data  Collection  Methods.  Having  defined  data  requirements  in 
step  2,  and  having  suggested  some  modest  data  processing  approaches,  in 
step  3  we  must  define  data  collection  methods.  From  a  technological 
point  of  view,  collection  of  position  location  information  is  our  great¬ 
est  problem.  The  problem  is  very  critical  because  position  location, 
i.e.,  tactical  trajectory,  is  the  foundation  of  the  HSI/ARI  data  sub¬ 
system.  And  without  good  position  data,  that  subsystem  is  a  house  of 
cards . 

When  we  began  researching  the  ES  evaluation  problem  several  years 
ago,  we  anticipated  access  to  Army  instrumented  ranges.  We  soon  dis¬ 
covered  that  such  ranges  were  few,  far  between,  expensive  to  operate, 
and  mostly  unavailable.  Consequently  we  began  a  small  study  of  low-cost 
portable  alternatives  to  such  facilities  as  those  at  the  Combat  Develop¬ 
ments  Experimentation  Coiranand  and  at  Fort  Hood.  The  study  was  particu¬ 
larly  geared  to  supporting  the  Fort  Carson  test.  The  study,  by  Behavior 
Technology  Consultants,  Inc.,  looked  at  optical  triangulation,  optical 
ranging,  unattended  ground  sensors,  and  a  number  of  radio-ranging  tech¬ 
niques.  It  recommended  a  radar-ranging  system,  which  was  portable, 
relatively  low-cost,  and  sufficiently  accurate  but  which  was  still  be¬ 
yond  our  resources  (and  which  could  not  be  put  together  in  time  for 
Fort  Carson  anyway) .6  Hence  we  developed  some  labor-intensive  strate¬ 
gies  for  plotting  tactical  movements.  These  involved  intensive  map 
and  terrain  training  of  data  collectors,  and  systematic  cross-checking 
of  results.  In  addition,  the  HSI  methodology  included  some  techniques 
for  screening  bad  position  data  and  estimating  missing  points. 

If  we  can  succeed  in  adequately  solving  the  dilemmas  posed  by  steps 
1,  2,  and  3  of  an  ES  evaluation  system,  we  will  have  achieved  a  great 
deal.  However,  we  will  still  not  have,  in  my  opinion,  a  criterion- 
referenced  system.  Being  able  to  collect  various  kinds  of  process  and 
product  data  and  being  able  to  relate  these  data  to  each  other  are  criti¬ 
cal  issues.  But  their  interpretation  and  usefulness  for  training  diag¬ 
nosis  are  incomplete  without  performance  benchmarks  or  standards.  The 
problem  of  standards  has  been  avoided  for  two-sided  combat  training 
exercises  because  such  exercises  are  situation-specific  and  involve  a 
complex  and  not  very  well-understood  set  of  variables. 

4 .  Performance  Standards .  Accordingly,  we  have  underway  a  basic 
research  program  to  explore  the  standards  problem.  This  effort  stems 

in  part  from  a  model  developed  several  years  ago  by  Litton  under  contract 
to  ARI.  This  model,  the  Unit  Performance  Assessment  Model  (UPAM) ,  used 
the  policy  capture  technique  to  generate  indices  of  combat  proficiency. 


O'Heeron,  M.  K. ,  Howell,  W.  Y.,  Frazier,  T.  W.,  &  Johnson,  E.  Field 
Measurement  and  Data  Collection  System  for  Engagement  Simulation  Field 
Exercises.  Final  Report.  Behavior  Technology  Consultants,  Inc., 

1  October  1977. 


The  index  values  resulted  from  a  linear  combination  of  variables,  mostly 
reflecting  various  casualty  measures.  Commanders'  forecasts  of  these 
measures  for  an  upcoming  exercise  were  to  provide  benchmarks  against 
which  actual  data  were  to  be  compared. 

A  more  recent  modeling  effort  is  called  Combat  Operations  Training 
Effectiveness  Analysis  (COTEAM) .  This  effort  picks  up  on  the  concept  of 
situation-specific  forecasting  to  provide  performance  benchmarks,  but 
develops  the  concept  further  in  some  ways  we  think  are  c[uite  significant. 
COTEAM  hopes  to  do  several  key  things: 

1.  Define  methods  for  forecasting  products,  processes,  and  dynam¬ 
ics  of  ES  exercises  in  a  situation-specific  context.  The  UPAM 
system  required  forecasting  but  left  the  problem  up  to  the  CO's 
own  devices . 

2.  Separate  nontraining  effects  such  as  terrain,  mission  type,  and 

force  ratios,  from  training  effects  by  addressing  two  kinds  of 
benchmarks:  training-system  referenced  and  combat  referenced. 

The  curve  in  Figure  7  suggests  why  two  types  of  benchmarks  are 
needed.  Imagine  that  you  could  unambiguously  define  training  system 
benchmarks  (1-5)  representing  various  points  in  a  training  cycle,  e.g., 

1  =  entry  level  performance,  5  =  final  stage  in  unit  training.  Now 
imagine  that  you  could  define  various  sets  of  operational  conditions 
(Sj^,  S2,  S2)  where  such  variables  as  weapon  mix,  terrain  type,  doctrine, 
force  ratio  define  the  sets.  Further  imagine  that  we  could  generate 
performance  curves  as  functions  of  training  level  and  operational  con¬ 
dition  set.  What  conclusions  could  you  draw  from  Figure  7? 

1.  The  conditions  of  S]^  are  such  that  training  effects  are  com¬ 
pletely  overwhelmed.  As  a  training  manager  you  would  avoid 
S]^  because  it  does  not  allow  for  a  differentiation  across 
levels  of  training. 

2.  S2  and  S3,  on  the  other  hand,  would  both  be  potentially  useful 
to  the  training  manager. 

3.  The  individual  who  sits  above  the  training  manager  and  balances 
training,  force  development,  and  doctrine  would  probably  exclude 
both  S3  and  S2  as  potential  training  conditions,  because  neither 
set  of  conditions  permits  training  to  bring  combat  units  up  to 
an  acceptable  level  of  readiness. 

There  are  at  least  five  methods  for  generating  the  benchmark  curves: 

1.  ARTEP  evaluator/controller  (E/C)  estimations 

2.  Use  of  the  Delphi  technique  by  military  experts 
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Figure  7.  Example  of  performance  benchmarks  referenced  to  training  system  and  combat. 


3.  Board  games 

4.  Analytic  math  models  such  as  Lancaster  and  HERO 

5.  Computer  simulation. 

We  developmentally  tested  the  first  method,  E/C  estimation,  duri.ng 
a  validation  of  rifle  squad  engagement  simulation  in  April  1977  at  Fort 
Ord,  Calif.  Figures  8-11  document  this  pilot  effort.^ 

Figure  8  shows  the  scenario  and  instructions  that  were  given  to 
subjects  for  a  squad  movement  to  contact.  The  subjects  were  NCO's  act¬ 
ing  as  squad  leaders . 

Figure  9  shows  the  ki.nds  of  forecasting  that  the  NCO's  did,  and  the 
data  that  resulted.  Figures  10  and  11  show  scenario,  i.nstruction ,  and 
resulting  data  for  a  hasty  defense.  Generally,  our  impression  was  that 
forecasting  could  be  done  with  some  reliability  and  that  the  task  of 
forecasting  for  different  assuiriad  training  levels  was  not  insurmount¬ 
able.  Our  subjects  did  seam  to  be  able  to  discriminate  expected  tacti¬ 
cal  performance  across  assumed  training  levels. 

The  third  method,  board  gaming,  was  explored  during  the  combined 
arms  test  of  engage.ment  simulation.S  company-level  game  was  specially 
designed  with  ES  rules,  using  measures  being  ta.ken  during  concurrent 
field  exercises.  Results  of  the  two  sets  of  measurements  were  very  simi¬ 
lar  and  therefore  provided  preliminary  support  for  the  usefulness  of 
games  as  techniques  for  generating  baseline  data . 

This  paper  provides  only  a  broad  and  very  surfacial  view  of  a  com¬ 
plex  research  program;  it  does  not  reveal  the  scope  of  effort  invo].ved. 
Some  indication  of  the  size  of  the  effort  is  its  staffing.  Approximately 
16  ARI  behavioral  scientists  with  advanced  degrees  are  partially  or  fully 
involved  with  the  program.  They  are  supported' by  the  services  of  four 
private  behavioral  research  companies.  The  very  active  and  indispensable 
support  of  our  TRA.DOC  sponsors  probably  adds  another  5  professional 
man-years . ^ 


"^The  materials  in  Figures  8-11  were  prepared  and  administered  by 
Dr.  F.  H.  Steinheiser  of  ARI. 

g 

Medlin,  S.  M.  Behavioral  Forecasting  for  REALTRAIN  Combined  Arms. 
ARI  Technical  Paper  in  preparation. 

9 

Our  sponsors  are  the  Training  System  Manager  for  Tactical  Engagement 
Simulation  Systems,  Fort  Eustis,  Va. ,  and  The  Directorate  of  Training 
Developments,  Fort  Knox,  Ky. 
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What  the  payoff  for  this  effort  will  be,  we  cannot  predict.  But 
its  significance  lies  at  least  partially  in  its  potential  contribution 
to  the  Army's  proposed  National  Training  Center  at  Fort  Irwin,  Calif. 
Large  sums  of  money  are  likely  to  be  invested  in  the  production  of  a 
very  sophisticated  instrumented  range,  capable  of  generating  enormous 
quantities  of  high  resolution  data.  If  the  Army's  capacity  to  select, 
process,  and  interpret  those  data  for  training  purposes  does  not  match 
its  capacity  to  supply  the  hardware  and  engineering  involved  in  instru¬ 
menting  a  range,  the  Fort  Irwin  concept  may  not  reach  its  full  potential 


10  man  attacking  squad  (testing  squad) 


4  man  (standard)  defense 

Scenario :  A  ten-man  squad  is  the  point  element  of  the  platoon  in  a 
movement  to  contact .  The  squad  will  know  that  they  can  expect  con¬ 
tact  at  any  moment.  They  will  have  just  crossed  a  danger  area  where 
they  encountered  sniper  fire,  without  taking  any  losses.  The  squad 
is  now  approaching  an  enemy  OP,  consisting  of  four  men  with  a  machine 
gun  in  well  concealed  positions .  Time  t  =  0  occurs  as  the  squad 
clears  the  danger  area. 


Instructions 

Your  own  opinions  and  estimates  are  being  requested.  This  is  NOT  a 
test  of  your  personality;  the  data  will  be  used  strictly  for  scientific 
purposes. 

Assume  that  all  members  of  your  squad  have  only  been  through  Basic 
Combat  Training  (BCT) .  Now  on  the  next  two  pages,  go  down  the  first 
(BCT)  column,  and  put  your  answer  in  each  box  for  each  question.  If 
a  more  detailed  answer  is  called  for,  use  the  reverse  side  of  the  paper. 

Now  assume  that  your  squad  has  recently  passed  Infantry  Level  2  ARTEP, 
shown  in  column  two.  Answer  each  of  the  questions  again  for  this  column. 

Assuming  three  days  of  SCOPES  training,  answer  the  questions  again  in 
the  third  column. 

Finally,  assume  that  all  members  of  your  squad  are  combat  experienced 
Rangers,  and  answer  all  questions  in  the  }DOxes  for  the  fourth  column. 


Figure  8.  Movement  to  contact  against  an  OP. 
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Level  of  Training  of  Tested  Squad 
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Figure  9.  Forecasts  of  infantry  squad  movement  to  contact. 
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Figure  9.  Forecasts  of  infantry  squad  movement  to  contact  (cont) . 


7  man  attacking  squad  (controlled  aggressor) 
10  man  defending  squad  (tested  squad) 


Scenario;  A  ten-man  squad  established  a  hasty  defense  as  part  of  a 
larger  platoon  defensive  perimeter.  They  will  have  approximately  15 
minutes  from  the  delivery  of  the  frag  order  to  establish  the  hasty 
defense.  At  that  time  an  enemy  counterattack,  consisting  of  7  men 
with  a  machinegun  will  begin  their  approach  toward  the  defensive 
positions.  The  counterattack  movement  will  begin  at  a  position 
approximately  100  meters  from  the  defense.  Time  t  =  0  occurs  with 
the  delivery  of  the  frag  order. 


Instructions 

Your  own  opinions  and  estimates  are  being  requested.  This  is  NOT  a 
test  of  your  personality;  the  data  will  be  used  strictly  for  scientific 
purposes . 

Assume  that  all  members  of  your  squad  have  only  laeen  through  Basic 
Combat  Training  (BCT) .  Now  on  the  next  two  pages,  go  down  the  first 
(BCT)  column,  and  put  your  answer  in  each  box  for  each  question.  If 
a  more  detailed  answer  is  called  for,  use  the  reverse  side  of  the  paper. 

Now  assume  that  your  squad  has  recently  passed  Infantry  Level  2  ARTEP, 
shown  in  column  two.  Answer  each  of  the  questions  again  for  this  column. 

Assuming  three  days  of  SCOPES  training,  answer  the  questions  again  in 
the  third  column. 

Finally,  assume  that  all  members  of  your  squad  are  combat  exT>erienced 
Rangers,  and  answer  all  questions  in  the  boxes  for  the  fourth  column. 


Figure  10.  Hasty  defense. 
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value  is  the  mean  of  8  respondents;  *  is  the  standard  deviation  from  that  mean 


